Multi-modality multi-scale cardiovascular disease subtypes classification using Raman image and medical history

Raman spectroscopy (RS) has been widely used for disease diagnosis, e.g., cardiovascular (CVD), owing to its efficiency and component-specific testing capabilities. A series of popular deep learning methods have recently introduced learn nuance features from RS binary classifications achieved outstanding performance than conventional machine methods. However, these existing still confront some challenges in classifying subtypes CVD. For example, the between is quite hard capture represent by intelligent models due chillingly similar shape sequences. Moreover, medical history information an essential resource distinguishing subtypes, but they are underutilized. In light this, we propose a multi-modality multi-scale model called M3S, which novel method with two core modules address issues. First, convert data various resolution images Gramian angular field (GAF) enlarge nuance, two-branch structure leveraged get embeddings distinction feature extraction module. Second, probability matrix weight enhance classification capacity combining fusion We perform extensive evaluations M3S found on our in-house dataset, accuracy, precision, recall, specificity, F1 score 0.9330, 0.9379, 0.9291, 0.9752, 0.9334, respectively. These results demonstrate that high robustness compared diagnosing CVD subtypes.